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DETAILED ACTION 

1 . This action is responsive to the amendment filed on 5/4/2005. 
This action is made Final. 

2. In the amendment, claims 1,3-21 are pending in the case. Claims 1, and 18 are 
independent claims. 

3. The rejections of claims 1, 3-4, and 6-21 rejected under 35 U.S.C. 103(a) as being 
unpatentable over Uppaluru (Pat. # 6,400,806 Bl, 6/4/2002, filed on 4/5/1999) have been 
withdrawn as necessitated by the amendment. 

4. The rejection of claim 5 rejected under 35 U.S.C. 103(a) as being unpatentable over 
Uppaluru, in view of Goldhor (Pat. # 5,101,375, 3/31/1992) has have been withdrawn as 
necessitated by the amendment. 

Drawings 

5. The drawings filed on 9/22/2000 have been approved by the Examiner. 

Claim Objections 

6. The objection to claim 21 has been withdrawn as necessitated by an appropriate 
amendment. 
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Claim Rejections - 35 USC§103 

7. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in section 
102 of this title, if the differences between the subject matter sought to be patented and the prior art are such that the 
subject matter as a whole would have been obvious at the time the invention was made to a person having ordinary skill 
in the art to which said subject matter pertains. Patentability shall not be negatived by the manner in which the 
invention was made. 

8. Claims 1, 3-4, and 6-21 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Uppaluru (Pat. # 6,400,806 Bl, 6/4/2002, filed on 4/5/1999), in view of Lemay et al, hereinafter 
Lemay, "Laura Lemay's Web Workshop JavaScript", Sams.net, 1996, pp.1 16-1 18, 130-139. 

Regarding independent claim 1, Uppaluru discloses the use of a conventional browser, 
which is modified with appropriate voice information extensions using HVML (Hyper Voice 
Markup Language) for displaying and playing web pages, such as web forms — navigating forms. 
The user can navigate or interact with the voice web pages using the mouse, and microphone — 
verbal, and tactile interaction—. Using tags, a user can also supply input, such as spoken 
alphabet, and digit, keyword, proper names, and fi"ee-form voice information input into HVML 
forms, for the purpose of filling in these forms— navigating form fields (having variable content 
and form fields), and submitting to an agent for processing — mutimodal browser for submission 
to an application- (col.6, lines 53-57, col.8, line 2-col.9, line 6, col. 10, line 34-col.l 1, line 14). 

Moreover, Uppaluru discloses a user's web browser for accessing, and navigating forms 
requested, and supplied fi-om a server over the Internet (col.8, line 2-col.9, line 6). 
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Moreover, Uppaluru discloses prompting for the input of information into a web page, 
such as a calendar form using the conventional browser. The input originates from a mouse, 
microphone, etc. — verbal or tactile (coL8, line 2-col.9, line 6, col. 12, lines 20-67). 

Further, Uppaluru discloses prompting for the input of information into a web page, such 
as database entry, query forms, a calendar form (day, month, year information), business white 
pages form (company name, city, state code information), using the conventional browser, and 
input from mouse, microphone, etc. — verbal or tactile (coL8, line 2-col.9, line 6, col.lO, lines 
34-67, col. 12, lines 6-67). Uppaluru fails to explicitly teach the moving to another form field 
requiring user provided input either after a current form field has been filled in by the user or 
the user selects by verbal or tactile interaction another form field. However, it would have been 
obvious to a person of ordinary skill in the art at the time of the invention to have moved from 
one field to another, because Uppaluru teaches above the filling in of forms by providing 
requested user input, which provides the benefit of supplying information necessary to quickly, 
and smoothly retrieve web pages from the web server using voice and/or visual input. 

Further, Uppaluru teaches the inputting of a company's partial information, such as 
company name, city, state code information, into the voice web query forms, and retrieving 
information from a database over the Internet, such as company's complete information using 
response pages, which are presented as a result of the partial submission of information (col. 10, 
lines 34-coLl 1, line 14, col. 12, lines 1 1-67). In other words, once the form is filled in it is 
submitted to the server and the complete information is retrieved and sent within a presentation 
page, Uppaluru fails to explicitly teach the using a command to submit the form after the user 
has supplied input for all required fields. However, Lemay teaches a submit button for 
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submitting a form (pages 116-117, fig.6.3)- using a command to submit the form. It would have 
been obvious to a person of ordinary skill in the art at the time of the invention to combine 
Uppaluru, and Lemay, because Uppaluru teaches above the filling in of forms by providing 
requested user input, which provides the benefit of supplying information necessary to quickly, 
and smoothly retrieve web pages from the web server using the submit button, and voice and/or 
visual input. 

Furthermore, Uppaluru fails to explicitly teach the wherein said multi-modal browser 
continues to prompt the user until the form is completed. However, Lemay teaches an error 
fimction for displaying an error message(s) whenever a criteria for filling out a form field is not 
met, and focusing on the form field so that the proper information is input into the form fields 
(pages 134-137, fig.6.6). In other words, an error for the field continues to be presented until the 
appropriate criteria is met— browser continues to prompt the user until the form is completed. It 
would have been obvious to a person of ordinary skill in the art at the time of the invention to 
combine Uppaluru, and Lemay, because Lemay teaches enabling the user to receive immediate 
feedback without waiting on the server (page 132, parag. 1-3), which provides the benefit of to 
quickly informing the user, and saving time needed to fill out the voice form. 

Regarding claim 3, which depends on claim 1, Uppaluru discloses a voice form 
prompting — reading aloud- for the input of information — heading- into a web page form, such 
as a calendar form using the conventional browser, and inputs from mouse, microphone, etc. The 
voice form is submitted once required information has been input — audibly completing (col. 10, 
lines 34-col.ll, line 14, col.8, line 2-col.9, line 40, col. 12, lines 20-67). 
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Regarding claim 4, which depends on claim 3, Uppaluru discloses the entering of a 
"REVIEW" — command" selection for reviewing form values within a voice form (coL25, lines 
61-67). 

Regarding claim 6, which depends on claim 1, Uppaluru discloses the web browser 
responds to user's voice command by matching them with personaUzed vocabulary — the 
browser responds to one or more verbal commands (col.8, line 2-coL9, line 6, col.lO, lines 34- 
67, col. 12, lines 6-67, col. 18, lines 29-50). 

Regarding claim 7, which depends on claim 6, Uppaluru discloses the entering of a 
"SKIP" — command" selection for skipping a form value within a voice form (col.25, lines 61- 
67, col.8, lines 63-67). 

Moreover, Uppaluru discloses the entering of a "REVIEW" — command- selection for 
reviewing form values within a voice form (col.25, lines 61-67). Uppaluru fails to expUcitly 
teach a command that directs the browser to review the form to ensure that all fields contain 
information. However, it would have been obvious to a person of ordinary skill in the art at the 
time of the invention to have reviewed the form to ensure information was present in all fields, 
because Uppaluru teaches above the reviewing of all the values in a voice form, which provides 
the benefit of supplying appropriate information in the forms so as to retrieve information using 
the form without incurring an error, which would also save time needed in refilling the wrong 
values in the form. 
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Moreover, Uppaluru discloses the entering of a "submit" — command- selection for 
submitting the form to a server (col.25, lines 50-67.). 

Further, Uppaluru discloses the entering of a "reset" — command- selection for reverting 
to the original default values of the form — canceling information currently within a field (col.25, 
lines 46-67). 

Furthermore, Uppaluru discloses the entering of a "reload" — command- selection for 
reloading a form (col.25, lines 46-67). Uppaluru fails to explicitly teach a command that directs 
the browser to clear the firom and reprocess it. However, it would have been obvious to a person 
of ordinary skill in the art at the time of the invention to reprocess the reloaded form, because 
Uppaluru teaches above the submitting of information to a server, which provides the benefit of 
supplying a form in accordance to a user's input, so as to provide the correct information to the 
server. 

Regarding claim 8, which depends on claim 1, Uppaluru discloses a voice form for 
guiding a user step by step — default mode in which order in which they are presented on the 
form- on supplying needed information (col.21, lines 7-67, col.23, lines 50-60). 

Regarding claim 9, which depends on claim 1, Uppaluru discloses a "PAUSE 
TIMEOUT" tag which allows the browser to pause until the user inputs or a set time period 
elapses (col.24, lines 56-67). Uppaluru fails to explicitly teach prompting the user for input by 
the browser after a specified time period if the user has not responded to an earlier prompt. 
However, it would have been obvious to a person of ordinary skill in the art at the time of the 
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invention to prompt the user again for input, because Uppaluru teaches above termination of 
input standby if the time expires, so when an input is mandatory for the field, this combination 
would provide the benefit of obtaining the mandatory input, and avoiding error triggered by not 
having all the necessary input. 

Regarding claim 10, which depends on claim 1, Uppaluru discloses a web browser voice 
output — audio queue- for prompting and playing voice strings in the order they are found in a 
web page (one right after the other) — prompting and moving through web page--, and using 
commands for terminating or exiting the processing of the tag (col.8, line 2-col.9, line 6, col. 10, 
lines 34-67, col.l2, lines 20-67, coL23, lines 39-67, col.24, lines 53-67). 

Regarding claim 11, which depends on claim 10, Uppaluru discloses a web browser voice 
output — audio queue- for prompting, and playing voice strings — text to be spoken- in the 
order they are found in a web page (one right after the other) (col.8, line 2-col.9, line 6, col. 10, 
lines 34-67, col.l2, lines 20-67, coL23, lines 39-67). 

Regarding claim 12, which depends on claim 10, Uppaluru discloses a web browser voice 
output — audio queue- for prompting, and playing voice strings in the order they are found in a 
web page (one right after the other). The 'Velcome" tag indicates entry to the form, and the 
"<VoiceString>" indicates an exit of the form (col.8, line 2-col.9, line 6, col.lO, lines 34-67, 
col.12, lines 20-67, col 23, lines 39-67). 
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Regarding claim 13, which depends on claim 10, Uppaluru discloses a web browser voice 
output — audio queue- for playing voice strings in the order they are foxmd in a web page (one 
right after the other). The "TERMINATE" attribute indicates exit from a form field or element 
(col. 10, lines 34-67, col.23, lines 39-67). 

Regarding claim 14, which depends on claim 10, Uppaluru discloses a web browser voice 
output — audio queue— for prompting, and playing voice strings in the order they are found in a 
web page (one right after the other). The "PAUSE TIMEOUT" attribute for pausing indefinitely 
(if a value is 0) for a user until an input is made — interruptible pause to the audio (col. 10, lines 
34-67, col.23, lines 39-67). 

Regarding claim 15, which depends on claim 10, Uppaluru discloses a web browser voice 
output — audio queue- for replaying voice strings in the web page using a "reload" command — 
repositioning of the audio queue (col. 10, lines 34-67, col.23, lines 53-67). 

Regarding claim 16, which depends on claim 15, Uppaluru discloses a web browser voice 
output — audio queue— for replaying voice strings in the web page using a "reload" command, 
which replays the page starting at the beginning (col.lO, lines 34-67, col.23, lines 53-67). In 
other words if the page is half-way done and the user selects the reload command, then the voice 
output will start the reading of the web page starting over at the beginning — loop back and 
repeat part of the audio queue. 
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Regarding claim 17, which depends on claim 1, Uppaluru discloses that a web browser 
pauses after a voice statement until a specified input, such as voice input is made to an HVML 
form — accepting input by verbal interaction in response to said prompting step— (col.8, lines 2- 
col.9, line 6, 9-40). 

Regarding independent claim 18, Uppaluru discloses the use of a conventional browser, 
which is modified with appropriate voice information extensions using HVML (Hyper Voice 
Markup Language) — mutimodal browser- for displaying and playing web pages, such as web 
forms — navigating forms. Using tags, a user can also supply input, such as spoken alphabet, and 
digit, keyword, proper names, and free-form voice information input into HVML forms on the 
browser, for the purpose of filling in these forms— navigating form fields (having variable 
content and form fields), and submitting to an agent for processing — mutimodal browser for 
submission to an application— (col.6, lines 1 1-57, col.8, line 2-col.9, lines 6, 62-coLlO, lines 23, 
34-col.ll,line 14). 

Moreover, Uppaluru discloses prompting for the input of information into a web page, 
such as a entry, query, calendar forms using the conventional browser. The input originates from 
a mouse, microphone, etc. — verbal or tactile (col.8, line 2-col.9, line 6, col. 12, lines 20-67). 

In addition, Uppaluru discloses that a web browser pauses after a voice statement until a 
specified input, such as voice input is made to an HVML form fields — accepting verbal 
responses from a user, and for entering those responses in said field ~ (col.8, lines 65-col.9, line 
6, 9-40). 
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Further, Uppaluru discloses the browser prompting for the input of information into a 
web page, such as database entry, query forms, a calendar form (day, month, year information), 
business white pages form (company name, city, state code information), using the conventional 
browser, and input from mouse, microphone, etc. — verbal or tactile (col.8, line 2-coL9, line 6, 
col. 10, lines 34-67, col. 12, lines 6-67). Uppaluru fails to explicitly teach the moving to another 
form field requiring user provided input either after a current form field has been filled in by the 
user or the user selects by verbal or tactile interaction another form field. However, it would 
have been obvious to a person of ordinary skill in the art at the time of the invention to have 
moved from one field to another, because Uppaluru teaches above the filling in of forms by 
providing requested user input, which provides the benefit of supplying information necessary to 
quickly, and smoothly retrieve web pages from the web server using voice and/or visual input. 

Further, Uppaluru teaches the inputting of a company's partial information, such as 
company name, city, state code information, into the voice web query forms, and retrieving 
information from a database over the Internet, such as company's complete information using 
response pages, which are presented as a result of the partial submission of information (col. 10, 
lines 34-col.l 1, line 14, col. 12, lines 1 1-67). In other words, once the form is filled in it is 
submitted to the server and the complete information is retrieved and sent within a presentation 
page. Uppaluru fails to explicitly teach the using a command to submit the form after the user 
has supplied input for all required fields. However, Lemay teaches a submit button for 
submitting a form (pages 116-117, fig.6.3)~ using a command to submit the form. It would have 
been obvious to a person of ordinary skill in the art at the time of the invention to combine 
Uppaluru, and Lemay, because Uppaluru teaches above the filling in of forms by providing 
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requested user input, which provides the benefit of supplying information necessary to quickly, 
and smoothly retrieve web pages fi-om the web server using the submit button, and voice and/or 
visual input. 

Furthermore, Uppaluru fails to explicitly teach the wherein said multi-modal browser 
continues to prompt the user until the form is completed. However, Lemay teaches an error 
function for displaying an error message(s) whenever a criteria for filling out a form field is not 
met, and focusing on the form field so that the proper information is input into the form fields 
(pages 134-137, fig,6.6). In other words, an error for the field continues to be presented xmtil the 
appropriate criteria is met— browser continues to prompt the user until the form is completed. It 
would have been obvious to a person of ordinary skill in the art at the time of the invention to 
combine Uppaluru, and Lemay, because Lemay teaches enabling the user to receive immediate 
feedback without waiting on the server (page 132, parag. 1-3), which provides the benefit of to 
quickly informing the user, and saving time needed to fill out the voice form. 

Regarding claims 19, which depends on claim 18, Uppaluru discloses a "PAUSE 
TIMEOUT" tag which allows the browser to pause until the user makes an input, such as a voice 
input, into HVML pages, such as the voice form, or a set time period elapses — timer which 
functions in conjunction with said mechanism for moving, for determining if a user has made a 
selection by a verbal response (col.24, lines 56-67, col.9, lines 1-7, col. 10, lines 34-54). 

Claim 20 is directed towards a computer-readable medium for performing the steps found 
in claim 1, and therefore is similarly rejected. 
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Regarding claim 21, which depends on claim 21, Uppalura discloses that a web browser 
pauses after a voice statement until a specified input, such as voice input is made to an HVML 
form fields — accepting input by verbal interaction in response to said prompting step (col. 8, 
lines 2-C0I.9, line 6, 9-40, col. 12, lines 20-67). 

9. Claim 5 is rejected xmder 35 U.S.C. 103(a) as being unpatentable over Uppaluru, in view 
of Lemay, and further in view of Goldhor (Pat. # 5,101,375, 3/31/1992). 

Regarding claim 5, which depends on claim 3, Uppaluru discloses the use of a 
conventional browser, which is modified with appropriate voice information extensions using 
HVML (Hyper Voice Markup Language) — mutimodal browser- for displaying and playing web 
pages, such as web forms. Using tags, a user can also supply input, such as spoken alphabet, and 
digit, keyword, proper names, and fi-ee-form voice information input into HVML forms for 
filling in these forms (col.6, lines 53-57, col.8, line 2-col.9, line 6, col. 10, line 34-col.ll, line 
14). Uppaluru fails to explicitly teach typing into the form fields words responsively spoken by 
the user. However, Goldhor teaches a speaker inserting words into spaces of a report form (coll, 
lines 26-67, col.3, lines 4-67, fig. 1-2). It would have been obvious to a person of ordinary skill in 
the art at the time of the invention to combine Uppaluru, and Goldhor, because Goldhor teaches 
providing the benefit of having much more details, using the speech system, than ordinary forms 
(col.4, lines 1-21). 
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Response to Arguments 
10. Applicant's arguments with respect to claims 1,3-21 have been considered but are moot 
in view of the new ground(s) of rejection. The Applicants indicate that the newly amended 
claims are not obvious over Uppaluru (pages 10-11). The Applicants are directed towards the 
newly added reference introduces in light of the amendment. 



Conclusion 

1 1 . Applicant's amendment necessitated the new ground(s) of rejection presented in 
this Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). 
Applicant is reminded of the extension of time policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS fi-om the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1.136(a) will be calculated fi-om the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS fi-om the date of this 
final action. 
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1. Any inquiry concerning this communication or earlier communications from the examiner 
should be directed to Cesar B. Paula whose telephone number is (571) 272-4128. The examiner 
can normally be reached on Monday through Friday from 8:00 a.m. to 4:00 p.m. (EST). 

If attempts to reach the examiner by telephone are unsuccessfiil, the examiner's 
supervisor, Stephen Hong, can be reached on (571) 272-4124. However, in such a case, please 
allow at least one business day. 

Information regarding the status of an application may be obtained from the Patent 

Application Retrieval (PAIR) system. Status inforaiation for published applications may be 

obtained from either Private PAIR or Public PAIR. Status information for unpublished 

appUcations is available through Private PAIR only. For more information about the PAIR 

system, go to http://portal.uspto.gov/extemal/portal/pair . Should you have any questions about 

access to the Private PAIR system, please contact the Electronic Business Center (EBC) at 866 

217-9197 (toll-free). 

Any response to this Action should be mailed to: 

Commissioner for Patents 
P.O. Box 1450 
Alexandria, VA 22313-1450 
Or faxed to: 

• (703) 703-872-9306, {(571)-273-8300 as of July 15, 2005} (for all Formal communications 
intended for entry) 
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PRIMARY EXAMINER 



